[
https://issues.apache.org/jira/browse/IMPALA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mostafa Mokhtar resolved IMPALA-6657.
-------------------------------------
Resolution: Fixed
Turns out tcmalloc is reason, issue fixe in 2.12
> Investigate why memory allocation in Exchange receiver node takes a long time
> -----------------------------------------------------------------------------
>
> Key: IMPALA-6657
> URL: https://issues.apache.org/jira/browse/IMPALA-6657
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Affects Versions: Impala 2.11.0
> Reporter: Mostafa Mokhtar
> Assignee: Mostafa Mokhtar
> Priority: Major
> Labels: memorymanager, performance, scalability
> Attachments: Impala query profile.txt
>
>
> It was observed while inserting large amounts of data into a Kudu table
> Exchange operator was running slow, query profile showed a big portion of the
> time was spent in memory allocation in the buffer pool
> {code}
> EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s853ms, % non-child:
> 0.23%)
> - ConvertRowBatchTime: 20s289ms
> - PeakMemoryUsage: 19.53 MB (20483562)
> - RowsReturned: 575.30M (575298780)
> - RowsReturnedRate: 27.10 K/sec
> Buffer pool:
> - AllocTime: 2h53m
> - CumulativeAllocationBytes: 261.26 GB (280526643200)
> - CumulativeAllocations: 13.70M (13697590)
> - PeakReservation: 18.06 MB (18939904)
> - PeakUnpinnedBytes: 0
> - PeakUsedReservation: 18.06 MB (18939904)
> - ReadIoBytes: 0
> - ReadIoOps: 0 (0)
> - ReadIoWaitTime: 0.000ns
> - WriteIoBytes: 0
> - WriteIoOps: 0 (0)
> - WriteIoWaitTime: 0.000ns
> RecvrSide:
> BytesReceived(8m32s): 20.91 GB, 37.03 GB, 45.62 GB, 53.22 GB,
> 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.30
> GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45
> GB, 121.30 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85
> GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29
> GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29
> GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29
> GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30
> GB, 148.30 GB, 148.30 GB
> - FirstBatchArrivalWaitTime: 1s071ms
> - TotalBytesReceived: 148.30 GB (159234237617)
> - TotalGetBatchTime: 5h53m
> - DataArrivalTimer: 5h52m
> SenderSide:
> - DeserializeRowBatchTime: 3h4m
> - NumBatchesArrived: 6.85M (6848795)
> - NumBatchesDeferred: 99.67K (99667)
> - NumBatchesEnqueued: 6.85M (6848795)
> - NumBatchesReceived: 6.85M (6848795)
> - NumEarlySenders: 0 (0)
> - NumEosReceived: 0 (0)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
