Qifan Chen has uploaded a new patch set (#31). ( 
http://gerrit.cloudera.org:8080/16474 )

Change subject: IMPALA-10178 Run-time profile shall report skews
......................................................................

IMPALA-10178 Run-time profile shall report skews

This fix addresses the current limitation in runtime profile that
skews existing in certain operators such as the rows read counter
(RowsRead) in the scan operators are not reported. A skew condition
exists when the number of rows processed at each operator instance
is not about the same and can be detected through standard deviation
(stddev). A high stddev (say > 5) usually implies the existence of
skew.

With the fix, such skew is detected for the following counters
  1. RowsRead in HDFS_SCAN_NODE and KUDU_SCAN_NODE
  2. ProbeRows and BuildRows in HASH_JOIN_NODE
  3. RowsReturned in GroupingAggregator, EXCHANGE and SORT_NODE

and reported as follows:
  1. In execution profile, a new skew summary that lists the names
     of the operators with skews;
  2. In the averaged profile for the corresponding operator, the list
     of values of the counter across all fragment instances in the
     backend processes, and the skew detection formula;
  3. Skew detection formula:
     a: stddev/mean > 0.05 and mean > 1,000,000, for conservative reporting
     b: stddev > 5, for aggressive reporting
  4. A new query option 'report_skew'
     = -1, 0: disable skew reporting
     = 1: enable conservative skew reporting (default)
     = 2: enable aggressive skew reporting

Examples of skews reported for a hash join and an hdfs scan.

In execution profile:

  ... ...
  skew(s) found at: HASH_JOIN_NODE (id=4), HDFS_SCAN_NODE (id=0)
  Per Node Peak Memory Usage: ...
  Per Node Bytes Read: ...
  Per Node User Time: ...
  Per Node System Time:
  ... ...

In averaged profiles(conservative reporting):

  HDFS_SCAN_NODE (id=2): ...
          Skew details: RowsRead ([2004992,1724693,2001351],
                                  stddev/mean=0.07, mean=1910345)

In averaged profiles (aggressive reporting):

  HASH_JOIN_NODE (id=4): ...
          Skew details: ProbeRows ([16904,17750,19197],
                                   stddev=946.77)

  HDFS_SCAN_NODE (id=0): ...
          Skew details: RowsRead ([913887,917913,1048604],
                                  stddev=62578.85)

Testing:
1. Added test_aggressive_skew_reporting_in_runtime_profile and
   test_conservative_skew_reporting_in_runtime_profile in
   test_observability.py to verify that the skews are reported.
2. Ran Core tests successfully.

Change-Id: I91041f2856eef8293ea78f1721f97469062589a1
---
M be/src/runtime/coordinator.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/runtime-profile-counters.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M be/src/util/stat-util.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M tests/query_test/test_hash_join_timer.py
M tests/query_test/test_observability.py
11 files changed, 295 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/16474/31
--
To view, visit http://gerrit.cloudera.org:8080/16474
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I91041f2856eef8293ea78f1721f97469062589a1
Gerrit-Change-Number: 16474
Gerrit-PatchSet: 31
Gerrit-Owner: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>

Reply via email to