[ 
https://issues.apache.org/jira/browse/IMPALA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mostafa Mokhtar resolved IMPALA-6657.
-------------------------------------
    Resolution: Fixed

Turns out tcmalloc is reason, issue fixe in 2.12

> Investigate why memory allocation in Exchange receiver node takes a long time
> -----------------------------------------------------------------------------
>
>                 Key: IMPALA-6657
>                 URL: https://issues.apache.org/jira/browse/IMPALA-6657
>             Project: IMPALA
>          Issue Type: Task
>          Components: Backend
>    Affects Versions: Impala 2.11.0
>            Reporter: Mostafa Mokhtar
>            Assignee: Mostafa Mokhtar
>            Priority: Major
>              Labels: memorymanager, performance, scalability
>         Attachments: Impala query profile.txt
>
>
> It was observed while inserting large amounts of data into a Kudu table 
> Exchange operator was running slow, query profile showed a big portion of the 
> time was spent in memory allocation in the buffer pool
> {code}
>       EXCHANGE_NODE (id=1):(Total: 5h53m, non-child: 48s853ms, % non-child: 
> 0.23%)
>            - ConvertRowBatchTime: 20s289ms
>            - PeakMemoryUsage: 19.53 MB (20483562)
>            - RowsReturned: 575.30M (575298780)
>            - RowsReturnedRate: 27.10 K/sec
>           Buffer pool:
>              - AllocTime: 2h53m
>              - CumulativeAllocationBytes: 261.26 GB (280526643200)
>              - CumulativeAllocations: 13.70M (13697590)
>              - PeakReservation: 18.06 MB (18939904)
>              - PeakUnpinnedBytes: 0
>              - PeakUsedReservation: 18.06 MB (18939904)
>              - ReadIoBytes: 0
>              - ReadIoOps: 0 (0)
>              - ReadIoWaitTime: 0.000ns
>              - WriteIoBytes: 0
>              - WriteIoOps: 0 (0)
>              - WriteIoWaitTime: 0.000ns
>           RecvrSide:
>             BytesReceived(8m32s): 20.91 GB, 37.03 GB, 45.62 GB, 53.22 GB, 
> 60.17 GB, 66.30 GB, 71.60 GB, 76.59 GB, 81.36 GB, 86.03 GB, 90.35 GB, 94.30 
> GB, 98.17 GB, 101.98 GB, 105.58 GB, 109.08 GB, 112.33 GB, 115.47 GB, 118.45 
> GB, 121.30 GB, 124.09 GB, 126.74 GB, 129.26 GB, 131.88 GB, 134.41 GB, 136.85 
> GB, 139.32 GB, 141.77 GB, 144.23 GB, 146.71 GB, 148.26 GB, 148.29 GB, 148.29 
> GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 
> GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 GB, 148.29 
> GB, 148.29 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 GB, 148.30 
> GB, 148.30 GB, 148.30 GB
>              - FirstBatchArrivalWaitTime: 1s071ms
>              - TotalBytesReceived: 148.30 GB (159234237617)
>              - TotalGetBatchTime: 5h53m
>                - DataArrivalTimer: 5h52m
>           SenderSide:
>              - DeserializeRowBatchTime: 3h4m
>              - NumBatchesArrived: 6.85M (6848795)
>              - NumBatchesDeferred: 99.67K (99667)
>              - NumBatchesEnqueued: 6.85M (6848795)
>              - NumBatchesReceived: 6.85M (6848795)
>              - NumEarlySenders: 0 (0)
>              - NumEosReceived: 0 (0)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

Reply via email to