[
https://issues.apache.org/jira/browse/IMPALA-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-5285:
----------------------------------
Summary: Account for missing significant contributors to query time in exec
summary (was: Entire time of query begin-end duration is not accounted for in
summary)
> Account for missing significant contributors to query time in exec summary
> --------------------------------------------------------------------------
>
> Key: IMPALA-5285
> URL: https://issues.apache.org/jira/browse/IMPALA-5285
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Mala Chikka Kempanna
> Priority: Major
> Labels: resource-management
>
> Entire time of query begin-end duration is not accounted for in summary(This
> is excluding client fetch wait timer)
> For eg
> 1) Start end time is
> Start Time: 2017-03-24 09:23:03.131144000
> End Time: 2017-03-24 10:54:57.702228000
> 2) Query exec summary and timelines look like below
> {code}
> Admission result: Admitted immediately
> ExecSummary:
> Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est.
> Peak Mem Detail
> ---------------------------------------------------------------------------------------------------------------------
> 04:EXCHANGE 1 17.320us 17.320us 0 1.18B 0
> -1.00 B UNPARTITIONED
> 03:AGGREGATE 18 4m35s 5m54s 0 1.18B 5.01 GB
> 5.99 GB FINALIZE
> 02:EXCHANGE 18 5s530ms 7s483ms 1.18B 1.18B 0
> 0 HASH(TRADEDATE,SYMB,EVENTID)
> 01:AGGREGATE 18 39s865ms 59s336ms 1.18B 1.18B 170.09 MB
> 56.95 GB STREAMING
> 00:SCAN HDFS 18 7s721ms 18s678ms 1.18B 1.18B 213.28 MB
> 264.00 MB cif20.dlereference
> - ComputeScanRangeAssignmentTimer: 835228
> - InactiveTotalTime: 0
> - TotalTime: 0
> Planner Timeline
> Analysis finished: 1768171
> Equivalence classes computed: 1953496
> Single node plan created: 2636132
> Runtime filters computed: 2764237
> Distributed plan created: 3178894
> Lineage info computed: 3297595
> Planning finished: 4567814
> Query Timeline
> Query submitted: 315704
> Planning finished: 8032272
> Submit for admission: 10787376
> Completed admission: 11053424
> Ready to start 37 fragment instances: 12622968
> All 37 fragment instances started: 591275200
> Rows available: 5514496400348
> First row fetched: 5514558703092
> Unregister query: 5514571011444
> ImpalaServer
> - ClientFetchWaitTimer: 68269916
> - InactiveTotalTime: 0
> - RowMaterializationTimer: 6332212
> - TotalTime: 0
> {code}
> I want to call attention to large discrepancy in total time accounted for in
> summary and timeline to one recorded ad start-end time
> This I believe is happening due to not recording fragment receive time in
> query profiles, which will remove the confusion when reading such profiles.
> This large time is listed under each fragment completion times in same profile
> {code}
> Averaged Fragment F01
> split sizes: min: 0, max: 0, avg: 0, stddev: 0
> completion times: min:1h31m max:1h31m mean: 1h31m stddev:2s911ms
> execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec
> stddev:0.00 /sec
> num instances: 18
>
> Averaged Fragment F00
> split sizes: min: 628.43 MB, max: 1.00 GB, avg: 827.75 MB, stddev:
> 104.93 MB
> completion times: min:35m18s max:1h31m mean: 1h4m stddev:13m10s
> execution rates: min:184.88 KB/sec max:303.77 KB/sec mean:224.86
> KB/sec stddev:30.22 KB/sec
> num instances: 18
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
