Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20507
Change subject: IMPALA-12460: Add lag and histogram of event processing in the log ...................................................................... IMPALA-12460: Add lag and histogram of event processing in the log This patch logs the lag of the event processing which is (latestEventTime - lastSyncedEventTime) at the end of processing an event batch. If the batch is slow to process, we also log the top-10 expensive events and top-10 targets contribute to this. Admins can decide whether to disable event processing on some tables. For table and partition level events, the target name is the table name. For db level events, the target name is the db name. For events that don't have db/table names, e.g. COMMIT_TXN, the target name is a constant string - "CLUSTER_WIDE". Also adds a thread annotation on the event-processor thread so we know what event is under processing using the jstacks. In the /metrics page, adds "pending-events" and "lag-time" in the event-processor metric group. The unit of the event time is second. This patch fixes some variable names that show them with ms. Tests: - Manually add some delay in event processing, verify the logs, jstacks and webUI. Change-Id: Ib9421b5e26bfa2324217ec9695fbd81636727d22 --- M be/src/util/event-metrics.cc M be/src/util/event-metrics.h M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java 5 files changed, 149 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/20507/1 -- To view, visit http://gerrit.cloudera.org:8080/20507 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib9421b5e26bfa2324217ec9695fbd81636727d22 Gerrit-Change-Number: 20507 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
