David Rorke created IMPALA-8537:
-----------------------------------
Summary: Negative values reported for
tmp-file-mgr.scratch-space-bytes-used under heavy spilling load
Key: IMPALA-8537
URL: https://issues.apache.org/jira/browse/IMPALA-8537
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.2.0
Reporter: David Rorke
Attachments: bad_spill_metrics.json
I'm running a workload that does a lot of spilling and noticed the value
reported for tmp-file-mgr.scratch-space-bytes-used is negative on all nodes.
Some details of the workload and cluster configuration:
* Generating a 10 TB TPC-DS partitioned parquet data set (very large sort).
* 30 impalads, each with 48 GB RAM and 14 scratch directories (each on a
separate drive)
* Rough estimate (based on query metrics) of total aggregate memory spilled
across the cluster since restart is 6.5 TB.
Snapshot of the bad metrics attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
