Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15154 )
Change subject: IMPALA-8712: Make ExecQueryFInstances async ...................................................................... Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc File be/src/runtime/coordinator-backend-state.cc: http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc@195 PS5, Line 195: rpc_latency_ I'm not sure if this variable is thread safe anymore - I think this Cb is being invoked by a kRPC thread right? but it gets read later on in Coordinator::FinishBackendStartup without holding onto the BackendState lock_ I think the same issue applies to exec_rpc_status_ It looks like reads of last_report_time_ms_ get around this by acquiring the lock_ inside the BackendState::last_report_time_ms() method. http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator-backend-state.cc@227 PS5, Line 227: SCOPED_TIMER(exec_serialization_timer_); move this to right before the ThriftSerializer is called? otherwise if the below code throws an error or returns, this could be non-zero, even if no serialization is done, which seems odd http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc@447 PS5, Line 447: ThriftSerializer serializer(true); should SCOPED_TIMER(exec_serialization_timer_); go here as well? http://gerrit.cloudera.org:8080/#/c/15154/5/be/src/runtime/coordinator.cc@982 PS5, Line 982: DCHECK(exec_rpcs_complete_.Load()) << "Exec() must be called first."; Will this break the ImpalaServer UnresponsiveBackendThread thread? It calls this method for each registered query, and if I'm reading the code in ImpalaServer::ExecuteInternal correctly, it register the query before calling Coordinator::Exec I think some of the other DCHECKs might suffer from the same issue - specifically any that are called due to a HTTP request from the Web UI. Since those are triggered asynchronously there could be a race condition between when the query is registered, when the HTTP request is fired, and when Coordinator::Exec is actually called. For example, BackendsToJson is called by ImpalaHttpHandler::QueryBackendsHandler. -- To view, visit http://gerrit.cloudera.org:8080/15154 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33ec96e5885af094c294cd3a76c242995263ba32 Gerrit-Change-Number: 15154 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Comment-Date: Thu, 13 Feb 2020 20:22:24 +0000 Gerrit-HasComments: Yes
