[ https://issues.apache.org/jira/browse/IMPALA-6257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Tauber-Marshall resolved IMPALA-6257. -------------------------------------------- Resolution: Cannot Reproduce Fix Version/s: Impala 2.11.0 This has not happened again AFAIK, and I've been unable to repro it, so closing for now. > TestKuduMemLimits::test_low_mem_limit_low_selectivity_scan times out in S3 > build > -------------------------------------------------------------------------------- > > Key: IMPALA-6257 > URL: https://issues.apache.org/jira/browse/IMPALA-6257 > Project: IMPALA > Issue Type: Bug > Affects Versions: Impala 2.11.0 > Reporter: Tianyi Wang > Assignee: Thomas Tauber-Marshall > Priority: Critical > Labels: broken-build > Fix For: Impala 2.11.0 > > > In a recent S3 build, > TestKuduMemLimits::test_low_mem_limit_low_selectivity_scan failed because a > fragment doesn't finish in time. The error message is: > {noformat} > 07:49:57 > TestKuduMemLimits.test_low_mem_limit_low_selectivity_scan[exec_option: > {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, > 'disable_codegen': False, 'abort_on_error': 1, > 'exec_single_node_rows_threshold': 0} | table_format: text/none-1] > 07:49:57 query_test/test_kudu.py:1062: in > test_low_mem_limit_low_selectivity_scan > 07:49:57 v.wait_for_metric("impala-server.num-fragments-in-flight", 0, > timeout=30) > 07:49:57 verifiers/metric_verifier.py:62: in wait_for_metric > 07:49:58 self.impalad_service.wait_for_metric_value(metric_name, > expected_value, timeout) > 07:49:58 common/impala_service.py:89: in wait_for_metric_value > 07:49:58 (metric_name, expected_value, timeout) > 07:49:58 E AssertionError: Metric value > impala-server.num-fragments-in-flight did not reach value 0 in 30s > {noformat} > Looking at the coordinator log, the query hangs for 30 seconds before it's > cancelled by the client: > {noformat} > I1127 07:30:37.677126 5939 coordinator.cc:789] Coordinator waiting for > backends to finish, 1 remaining > I1127 07:30:37.677865 5939 coordinator.cc:794] All backends finished > successfully. > I1127 07:30:37.677877 5939 coordinator.cc:1090] Release admssion control > resources for query 4044d47830c1cc98:8a63d27b00000000 > I1127 07:30:37.677942 6043 data-stream-mgr.cc:238] DeregisterRecvr(): > fragment_instance_id=4044d47830c1cc98:8a63d27b00000000, node=1 > I1127 07:30:37.677963 6043 data-stream-recvr.cc:235] cancelled stream: > fragment_instance_id_=4044d47830c1cc98:8a63d27b00000000 node_id=1 > I1127 07:30:37.678045 6043 query-state.cc:385] Instance completed. > instance_id=4044d47830c1cc98:8a63d27b00000000 #in-flight=1 status=OK > I1127 07:30:37.678073 6043 query-exec-mgr.cc:149] ReleaseQueryState(): > query_id=4044d47830c1cc98:8a63d27b00000000 refcnt=2 > I1127 07:31:08.084748 5939 impala-hs2-server.cc:699] CloseOperation(): > query_id=4044d47830c1cc98:8a63d27b00000000 > I1127 07:31:08.084779 5939 impala-server.cc:992] UnregisterQuery(): > query_id=4044d47830c1cc98:8a63d27b00000000 > I1127 07:31:08.084790 5939 impala-server.cc:1075] Cancel(): > query_id=4044d47830c1cc98:8a63d27b00000000 > I1127 07:31:08.084808 5939 coordinator.cc:895] Cancel() > query_id=4044d47830c1cc98:8a63d27b00000000 > I1127 07:31:08.084827 5939 coordinator.cc:905] CancelBackends() > query_id=4044d47830c1cc98:8a63d27b00000000, tried to cancel 0 backends > I1127 07:31:08.090203 5939 query-exec-mgr.cc:149] ReleaseQueryState(): > query_id=4044d47830c1cc98:8a63d27b00000000 refcnt=1 > I1127 07:31:08.101332 5939 impala-hs2-server.cc:381] CloseSession(): > request=TCloseSessionReq { > {noformat} > Compared to the same but successful query, the #in-flight differs: > {noformat} > I1128 07:35:45.863986 7571 coordinator.cc:789] Coordinator waiting for > backends to finish, 1 remaining > I1128 07:35:45.864734 7571 coordinator.cc:794] All backends finished > successfully. > I1128 07:35:45.864749 7571 coordinator.cc:1090] Release admssion control > resources for query 6f43945cef3a95be:744fd2a300000000 > I1128 07:35:45.864823 7671 data-stream-mgr.cc:238] DeregisterRecvr(): > fragment_instance_id=6f43945cef3a95be:744fd2a300000000, node=1 > I1128 07:35:45.864845 7671 data-stream-recvr.cc:235] cancelled stream: > fragment_instance_id_=6f43945cef3a95be:744fd2a300000000 node_id=1 > I1128 07:35:45.864953 7671 query-state.cc:385] Instance completed. > instance_id=6f43945cef3a95be:744fd2a300000000 #in-flight=0 status=OK > I1128 07:35:45.864982 7671 query-exec-mgr.cc:149] ReleaseQueryState(): > query_id=6f43945cef3a95be:744fd2a300000000 refcnt=2 > I1128 07:35:45.944633 7571 impala-hs2-server.cc:699] CloseOperation(): > query_id=6f43945cef3a95be:744fd2a300000000 > I1128 07:35:45.944655 7571 impala-server.cc:992] UnregisterQuery(): > query_id=6f43945cef3a95be:744fd2a300000000 > I1128 07:35:45.944666 7571 impala-server.cc:1075] Cancel(): > query_id=6f43945cef3a95be:744fd2a300000000 > I1128 07:35:45.944691 7571 coordinator.cc:895] Cancel() > query_id=6f43945cef3a95be:744fd2a300000000 > I1128 07:35:45.944710 7571 coordinator.cc:905] CancelBackends() > query_id=6f43945cef3a95be:744fd2a300000000, tried to cancel 0 backends > I1128 07:35:45.949873 7571 query-exec-mgr.cc:149] ReleaseQueryState(): > query_id=6f43945cef3a95be:744fd2a300000000 refcnt=1 > I1128 07:35:45.959841 7571 impala-hs2-server.cc:381] CloseSession(): > request=TCloseSessionReq { > {noformat} > In the query profile, there is a 30s gap between "First row fetched" (The > query only returns 1 row) and "Unregister query". > {noformat} > Query (id=4044d47830c1cc98:8a63d27b00000000): > DEBUG MODE WARNING: Query profile created while running a DEBUG build of > Impala. Use RELEASE builds to measure query performance. > Summary: > Session ID: c4db0e602700a43:81f5e7a5788465b0 > Session Type: HIVESERVER2 > HiveServer2 Protocol Version: V6 > Start Time: 2017-11-27 07:30:37.616861 > End Time: 2017-11-27 07:31:08.085625 > Query Type: QUERY > Query State: FINISHED > Query Status: OK > Impala Version: impalad version 2.11.0-SNAPSHOT DEBUG (build > 79a5de1028b1c9523d3a2525d2a189288ca55357) > User: jenkins > Connected User: jenkins > Delegated User: > Network Address: 127.0.0.1:34124 > Default Db: fkfp7v > Sql Statement: select * from tpch_kudu.lineitem limit 90 > Coordinator: impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22000 > Query Options (set by configuration): > ABORT_ON_ERROR=1,MEM_LIMIT=10485760,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0 > Query Options (set by configuration and planner): > ABORT_ON_ERROR=1,MEM_LIMIT=10485760,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0 > Plan: > ---------------- > Max Per-Host Resource Reservation: Memory=0B > Per-Host Resource Estimates: Memory=10.00MB > F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=0B mem-reservation=0B > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B > | > 01:EXCHANGE [UNPARTITIONED] > | limit: 90 > | mem-estimate=0B mem-reservation=0B > | tuple-ids=0 row-size=263B cardinality=90 > | > F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 > Per-Host Resources: mem-estimate=0B mem-reservation=0B > 00:SCAN KUDU [tpch_kudu.lineitem] > limit: 90 > mem-estimate=0B mem-reservation=0B > tuple-ids=0 row-size=263B cardinality=90 > ---------------- > Estimated Per-Host Mem: 10485760 > Per Host Min Reservation: > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22000(0) > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22001(0) > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22002(0) > Request Pool: default-pool > Admission result: Admitted immediately > ExecSummary: > Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. > Peak Mem Detail > ---------------------------------------------------------------------------------------------------------- > 01:EXCHANGE 1 14.999ms 14.999ms 90 90 0 > 0 UNPARTITIONED > 00:SCAN KUDU 3 12.666ms 12.999ms 270 90 1.52 MB > 0 tpch_kudu.lineitem > Errors: > Planner Timeline: 10.640ms > - Analysis finished: 2.106ms (2.106ms) > - Value transfer graph computed: 2.404ms (297.594us) > - Single node plan created: 8.894ms (6.490ms) > - Runtime filters computed: 9.062ms (167.506us) > - Distributed plan created: 9.165ms (103.670us) > - Planning finished: 10.640ms (1.474ms) > Query Timeline: 30s468ms > - Query submitted: 0.000ns (0.000ns) > - Planning finished: 12.999ms (12.999ms) > - Submit for admission: 12.999ms (0.000ns) > - Completed admission: 12.999ms (0.000ns) > - Ready to start on 3 backends: 12.999ms (0.000ns) > - All 3 execution backends (4 fragment instances) started: 13.999ms > (999.983us) > - Rows available: 47.999ms (33.999ms) > - First row fetched: 58.998ms (10.999ms) > - Unregister query: 30s468ms (30s409ms) > - ComputeScanRangeAssignmentTimer: 0.000ns > ImpalaServer: > - ClientFetchWaitTimer: 30s418ms > - RowMaterializationTimer: 1.999ms > Execution Profile 4044d47830c1cc98:8a63d27b00000000:(Total: 35.999ms, > non-child: 0.000ns, % non-child: 0.00%) > Number of filters: 0 > Filter routing table: > ID Src. Node Tgt. Node(s) Target type Partition filter Pending > (Expected) First arrived Completed Enabled > ------------------------------------------------------------------------------------------------------------------- > Backend startup latencies: Count: 3, min / max: 1ms / 1ms, 25th %-ile: > 1ms, 50th %-ile: 1ms, 75th %-ile: 1ms, 90th %-ile: 1ms, 95th %-ile: 1ms, > 99.9th %-ile: 1ms > Per Node Peak Memory Usage: > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22000(1.36 MB) > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22001(1.89 MB) > impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22002(1.33 MB) > - FiltersReceived: 0 (0) > - FinalizationTimer: 0.000ns > - NumBackends: 3 (3) > - NumFragmentInstances: 4 (4) > - NumFragments: 2 (2) > Averaged Fragment F01:(Total: 44.999ms, non-child: 0.000ns, % non-child: > 0.00%) > split sizes: min: 0, max: 0, avg: 0, stddev: 0 > completion times: min:45.999ms max:45.999ms mean: 45.999ms > stddev:0.000ns > execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec > stddev:0.00 /sec > num instances: 1 > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 80.18 KB (82103) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.36 MB (1428125) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 14.999ms > - TotalNetworkSendTime: 0.000ns > - TotalStorageWaitTime: 0.000ns > - TotalThreadsInvoluntaryContextSwitches: 0 (0) > - TotalThreadsTotalWallClockTime: 25.999ms > - TotalThreadsSysTime: 0.000ns > - TotalThreadsUserTime: 1.000ms > - TotalThreadsVoluntaryContextSwitches: 2 (2) > Fragment Instance Lifecycle Timings: > - ExecTime: 10.999ms > - ExecTreeExecTime: 0.000ns > - OpenTime: 14.999ms > - ExecTreeOpenTime: 14.999ms > - PrepareTime: 17.999ms > - ExecTreePrepareTime: 0.000ns > PLAN_ROOT_SINK: > - PeakMemoryUsage: 0 > CodeGen:(Total: 17.999ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 16.999ms > EXCHANGE_NODE (id=1):(Total: 14.999ms, non-child: 0.000ns, % non-child: > 0.00%) > - ConvertRowBatchTime: 0.000ns > - PeakMemoryUsage: 0 > - RowsReturned: 90 (90) > - RowsReturnedRate: 6.00 K/sec > DataStreamReceiver: > - BytesReceived: 32.87 KB (33662) > - DeserializeRowBatchTimer: 0.000ns > - FirstBatchArrivalWaitTime: 14.999ms > - PeakMemoryUsage: 48.90 KB (50074) > - SendersBlockedTimer: 0.000ns > - SendersBlockedTotalTimer(*): 0.000ns > Coordinator Fragment F01: > Instance 4044d47830c1cc98:8a63d27b00000000 > (host=impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22000):(Total: > 44.999ms, non-child: 0.000ns, % non-child: 0.00%) > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 80.18 KB (82103) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.36 MB (1428125) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 14.999ms > - TotalNetworkSendTime: 0.000ns > - TotalStorageWaitTime: 0.000ns > - TotalThreadsInvoluntaryContextSwitches: 0 (0) > - TotalThreadsTotalWallClockTime: 25.999ms > - TotalThreadsSysTime: 0.000ns > - TotalThreadsUserTime: 1.000ms > - TotalThreadsVoluntaryContextSwitches: 2 (2) > Fragment Instance Lifecycle Timings: > - ExecTime: 10.999ms > - ExecTreeExecTime: 0.000ns > - OpenTime: 14.999ms > - ExecTreeOpenTime: 14.999ms > - PrepareTime: 17.999ms > - ExecTreePrepareTime: 0.000ns > PLAN_ROOT_SINK: > - PeakMemoryUsage: 0 > CodeGen:(Total: 17.999ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 16.999ms > EXCHANGE_NODE (id=1):(Total: 14.999ms, non-child: 0.000ns, % > non-child: 0.00%) > - ConvertRowBatchTime: 0.000ns > - PeakMemoryUsage: 0 > - RowsReturned: 90 (90) > - RowsReturnedRate: 6.00 K/sec > DataStreamReceiver: > - BytesReceived: 32.87 KB (33662) > - DeserializeRowBatchTimer: 0.000ns > - FirstBatchArrivalWaitTime: 14.999ms > - PeakMemoryUsage: 48.90 KB (50074) > - SendersBlockedTimer: 0.000ns > - SendersBlockedTotalTimer(*): 0.000ns > Averaged Fragment F00:(Total: 34.332ms, non-child: 0.000ns, % non-child: > 0.00%) > split sizes: min: 0, max: 0, avg: 0, stddev: 0 > completion times: min:33.999ms max:37.999ms mean: 36.332ms > stddev:1.699ms > execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec > stddev:0.00 /sec > num instances: 3 > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 1.52 MB (1589936) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.53 MB (1600852) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 0.000ns > - TotalNetworkSendTime: 333.327us > - TotalStorageWaitTime: 27.666ms > - TotalThreadsInvoluntaryContextSwitches: 11 (11) > - TotalThreadsTotalWallClockTime: 38.999ms > - TotalThreadsSysTime: 1.333ms > - TotalThreadsUserTime: 6.664ms > - TotalThreadsVoluntaryContextSwitches: 15 (15) > Fragment Instance Lifecycle Timings: > - ExecTime: 12.666ms > - ExecTreeExecTime: 11.333ms > - OpenTime: 666.655us > - ExecTreeOpenTime: 666.655us > - PrepareTime: 20.999ms > - ExecTreePrepareTime: 0.000ns > DataStreamSender (dst_id=1):(Total: 999.982us, non-child: 0.000ns, % > non-child: 0.00%) > - BytesSent: 10.96 KB (11220) > - NetworkThroughput(*): 0.00 /sec > - OverallThroughput: 5.37 MB/sec > - PeakMemoryUsage: 688.00 B (688) > - RowsReturned: 90 (90) > - SerializeBatchTime: 666.655us > - TransmitDataRPCTime: 0.000ns > - UncompressedRowBatchSize: 24.06 KB (24637) > CodeGen:(Total: 19.666ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 18.999ms > KUDU_SCAN_NODE (id=0):(Total: 12.666ms, non-child: 0.000ns, % > non-child: 0.00%) > - BytesRead: 0 > - CollectionItemsRead: 0 (0) > - KuduRemoteScanTokens: 0 (0) > - NumScannerThreadsStarted: 3 (3) > - PeakMemoryUsage: 1.39 MB (1454080) > - RowsRead: 9.34K (9338) > - RowsReturned: 90 (90) > - RowsReturnedRate: 7.12 K/sec > - ScanRangesComplete: 1 (1) > - ScannerThreadsInvoluntaryContextSwitches: 10 (10) > - ScannerThreadsTotalWallClockTime: 25.666ms > - MaterializeTupleTime(*): 11.333ms > - ScannerThreadsSysTime: 666.333us > - ScannerThreadsUserTime: 5.998ms > - ScannerThreadsVoluntaryContextSwitches: 5 (5) > - TotalKuduScanRoundTrips: 2 (2) > - TotalReadThroughput: 0.00 /sec > Fragment F00: > Instance 4044d47830c1cc98:8a63d27b00000001 > (host=impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22002):(Total: > 35.999ms, non-child: 0.000ns, % non-child: 0.00%) > Hdfs split stats (<volume id>:<# splits>/<split lengths>): > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 1.33 MB (1395376) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.33 MB (1395376) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 0.000ns > - TotalNetworkSendTime: 999.982us > - TotalStorageWaitTime: 29.999ms > - TotalThreadsInvoluntaryContextSwitches: 27 (27) > - TotalThreadsTotalWallClockTime: 56.998ms > - TotalThreadsSysTime: 0.000ns > - TotalThreadsUserTime: 9.997ms > - TotalThreadsVoluntaryContextSwitches: 18 (18) > Fragment Instance Lifecycle Timings: > - ExecTime: 13.999ms > - ExecTreeExecTime: 12.999ms > - OpenTime: 0.000ns > - ExecTreeOpenTime: 0.000ns > - PrepareTime: 21.999ms > - ExecTreePrepareTime: 0.000ns > DataStreamSender (dst_id=1): > - BytesSent: 10.92 KB (11177) > - NetworkThroughput(*): 0.00 /sec > - OverallThroughput: 0.00 /sec > - PeakMemoryUsage: 688.00 B (688) > - RowsReturned: 90 (90) > - SerializeBatchTime: 0.000ns > - TransmitDataRPCTime: 0.000ns > - UncompressedRowBatchSize: 24.15 KB (24731) > CodeGen:(Total: 20.999ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 20.999ms > KUDU_SCAN_NODE (id=0):(Total: 12.999ms, non-child: 0.000ns, % > non-child: 0.00%) > - BytesRead: 0 > - CollectionItemsRead: 0 (0) > - KuduRemoteScanTokens: 0 (0) > - NumScannerThreadsStarted: 3 (3) > - PeakMemoryUsage: 1.32 MB (1386496) > - RowsRead: 12.01K (12014) > - RowsReturned: 90 (90) > - RowsReturnedRate: 6.92 K/sec > - ScanRangesComplete: 2 (2) > - ScannerThreadsInvoluntaryContextSwitches: 27 (27) > - ScannerThreadsTotalWallClockTime: 42.999ms > - MaterializeTupleTime(*): 12.999ms > - ScannerThreadsSysTime: 0.000ns > - ScannerThreadsUserTime: 8.997ms > - ScannerThreadsVoluntaryContextSwitches: 9 (9) > - TotalKuduScanRoundTrips: 3 (3) > - TotalReadThroughput: 0.00 /sec > Instance 4044d47830c1cc98:8a63d27b00000002 > (host=impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22000):(Total: > 34.999ms, non-child: 0.000ns, % non-child: 0.00%) > Hdfs split stats (<volume id>:<# splits>/<split lengths>): > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 1.33 MB (1395376) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.36 MB (1428125) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 0.000ns > - TotalNetworkSendTime: 0.000ns > - TotalStorageWaitTime: 27.999ms > - TotalThreadsInvoluntaryContextSwitches: 3 (3) > - TotalThreadsTotalWallClockTime: 25.999ms > - TotalThreadsSysTime: 1.999ms > - TotalThreadsUserTime: 2.999ms > - TotalThreadsVoluntaryContextSwitches: 13 (13) > Fragment Instance Lifecycle Timings: > - ExecTime: 12.999ms > - ExecTreeExecTime: 10.999ms > - OpenTime: 999.983us > - ExecTreeOpenTime: 999.983us > - PrepareTime: 20.999ms > - ExecTreePrepareTime: 0.000ns > DataStreamSender (dst_id=1):(Total: 999.983us, non-child: 0.000ns, % > non-child: 0.00%) > - BytesSent: 11.03 KB (11292) > - NetworkThroughput(*): 0.00 /sec > - OverallThroughput: 10.77 MB/sec > - PeakMemoryUsage: 688.00 B (688) > - RowsReturned: 90 (90) > - SerializeBatchTime: 999.983us > - TransmitDataRPCTime: 0.000ns > - UncompressedRowBatchSize: 24.05 KB (24623) > CodeGen:(Total: 20.999ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 18.999ms > KUDU_SCAN_NODE (id=0):(Total: 12.999ms, non-child: 0.000ns, % > non-child: 0.00%) > - BytesRead: 0 > - CollectionItemsRead: 0 (0) > - KuduRemoteScanTokens: 0 (0) > - NumScannerThreadsStarted: 3 (3) > - PeakMemoryUsage: 1.32 MB (1386496) > - RowsRead: 4.00K (4000) > - RowsReturned: 90 (90) > - RowsReturnedRate: 6.92 K/sec > - ScanRangesComplete: 1 (1) > - ScannerThreadsInvoluntaryContextSwitches: 3 (3) > - ScannerThreadsTotalWallClockTime: 11.999ms > - MaterializeTupleTime(*): 10.999ms > - ScannerThreadsSysTime: 1.999ms > - ScannerThreadsUserTime: 1.999ms > - ScannerThreadsVoluntaryContextSwitches: 2 (2) > - TotalKuduScanRoundTrips: 1 (1) > - TotalReadThroughput: 0.00 /sec > Instance 4044d47830c1cc98:8a63d27b00000003 > (host=impala-boost-static-burst-slave-10ec.vpc.cloudera.com:22001):(Total: > 31.999ms, non-child: 0.000ns, % non-child: 0.00%) > Hdfs split stats (<volume id>:<# splits>/<split lengths>): > - AverageThreadTokens: 0.00 > - BloomFilterBytes: 0 > - PeakMemoryUsage: 1.89 MB (1979056) > - PeakReservation: 0 > - PeakUsedReservation: 0 > - PerHostPeakMemUsage: 1.89 MB (1979056) > - RowsProduced: 90 (90) > - TotalNetworkReceiveTime: 0.000ns > - TotalNetworkSendTime: 1.000ns > - TotalStorageWaitTime: 24.999ms > - TotalThreadsInvoluntaryContextSwitches: 5 (5) > - TotalThreadsTotalWallClockTime: 33.999ms > - TotalThreadsSysTime: 2.000ms > - TotalThreadsUserTime: 6.998ms > - TotalThreadsVoluntaryContextSwitches: 14 (14) > Fragment Instance Lifecycle Timings: > - ExecTime: 10.999ms > - ExecTreeExecTime: 9.999ms > - OpenTime: 999.984us > - ExecTreeOpenTime: 999.984us > - PrepareTime: 19.999ms > - ExecTreePrepareTime: 0.000ns > DataStreamSender (dst_id=1):(Total: 1.999ms, non-child: 0.000ns, % > non-child: 0.00%) > - BytesSent: 10.93 KB (11193) > - NetworkThroughput(*): 0.00 /sec > - OverallThroughput: 5.34 MB/sec > - PeakMemoryUsage: 688.00 B (688) > - RowsReturned: 90 (90) > - SerializeBatchTime: 999.982us > - TransmitDataRPCTime: 0.000ns > - UncompressedRowBatchSize: 23.98 KB (24557) > CodeGen:(Total: 16.999ms, non-child: 0.000ns, % non-child: 0.00%) > - CodegenTime: 0.000ns > - CompileTime: 0.000ns > - LoadTime: 0.000ns > - ModuleBitcodeSize: 1.86 MB (1950540) > - NumFunctions: 0 (0) > - NumInstructions: 0 (0) > - OptimizationTime: 0.000ns > - PeakMemoryUsage: 0 > - PrepareTime: 16.999ms > KUDU_SCAN_NODE (id=0):(Total: 11.999ms, non-child: 0.000ns, % > non-child: 0.00%) > - BytesRead: 0 > - CollectionItemsRead: 0 (0) > - KuduRemoteScanTokens: 0 (0) > - NumScannerThreadsStarted: 3 (3) > - PeakMemoryUsage: 1.52 MB (1589248) > - RowsRead: 12.00K (12000) > - RowsReturned: 90 (90) > - RowsReturnedRate: 7.50 K/sec > - ScanRangesComplete: 2 (2) > - ScannerThreadsInvoluntaryContextSwitches: 2 (2) > - ScannerThreadsTotalWallClockTime: 21.999ms > - MaterializeTupleTime(*): 9.999ms > - ScannerThreadsSysTime: 0.000ns > - ScannerThreadsUserTime: 6.998ms > - ScannerThreadsVoluntaryContextSwitches: 5 (5) > - TotalKuduScanRoundTrips: 3 (3) > - TotalReadThroughput: 0.00 /sec > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)