[ https://issues.apache.org/jira/browse/IMPALA-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844526#comment-16844526 ]
Todd Lipcon commented on IMPALA-7799: ------------------------------------- Before reinventing this wheel please take a look at diagnostics_log.cc in Kudu. We do exactly this. > Store periodic snapshots of Impalad metrics > ------------------------------------------- > > Key: IMPALA-7799 > URL: https://issues.apache.org/jira/browse/IMPALA-7799 > Project: IMPALA > Issue Type: Improvement > Components: Backend > Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0 > Reporter: Michael Ho > Priority: Critical > > Currently, each Impala demon exposes a set of metrics exposed via the debug > webpage metrics page. While this may be very helpful for development, there > are many incidents in which one may want to record these metrics to do > postmortem analysis for various issues. > We should consider taking periodic snapshots of these Impalad metrics and > archive them for a certain retention period to allow for postmortem analysis. > We need to be mindful of the space usage concern (e.g. using compressed json > ?) > To enable easier analysis, one may need to build a tool (or use some existing > off-the-shell libraries) to show the collected snapshots as time series and > calculate various statistics (e.g. mean, median, min, max etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org