Oliver Draese created HIVE-21204: ------------------------------------ Summary: Instrumentation for read/write locks in LLAP Key: HIVE-21204 URL: https://issues.apache.org/jira/browse/HIVE-21204 Project: Hive Issue Type: Improvement Components: llap Reporter: Oliver Draese Assignee: Oliver Draese
LLAP has several R/W locks for serialization of updates to query tracker, file data, .... Instrumentation is added to monitor the * total amount of R/W locks within a particular category * average + max wait/suspension time to get the R/W lock A category includes all lock instances for particular areas (i.e. category is FileData and all R/W locks that are used in FileData instances are accounted within the one category). The monitoring/accounting is done via Hadoop Metrics 2, making them accessible via JMX. In addition, a new "locking" GET endpoint is added to the LLAP daemon's REST interface. It produces output like the following example: {{{}} {{ "statsCollection": "enabled",}} {{ "lockStats": [}} {{ {}}{{ "type": "R/W Lock Stats",}} {{ "label": "FileData",}} {{ "totalLockWaitTimeMillis": 0,}} {{ "readLock": {}} {{ "count": 0,}} {{ "avgWaitTimeNanos": 0,}} {{ "maxWaitTimeNanos": 0}} {{ },}} {{ "writeLock": {}} {{ "count": 0,}} {{ "avgWaitTimeNanos": 0,}} {{ "maxWaitTimeNanos": 0}} {{ }}} {{ },}} {{ { "}}{{type": "R/W Lock Stats",}} {{ "label": "QueryTracker",}} {{ "totalLockWaitTimeMillis": 0,}} {{ "readLock": {}} {{ "count": 0,}} {{ "avgWaitTimeNanos": 0,}} {{ "maxWaitTimeNanos": 0}} {{ },}} {{ "writeLock": {}} {{ "count": 0,}} {{ "avgWaitTimeNanos": 0,}} {{ "maxWaitTimeNanos": 0}} {{ }}} {{ } }}{{]}} {{ }}} {{}}} To avoid the overhead of lock instrumentation, lock metrics collection is disabled by default and can be enabled via the following configuration parameter: {{hive.llap.lockmetrics.collect = true}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)