karlovnv commented on issue #7000:
URL: https://github.com/apache/datafusion/issues/7000#issuecomment-2094206289
Another topic related issue is performance of **RowConverter** used for
grouping.
More than 75% of GroupedHashAggregateStream work is converting composite
aggregation key to row
Apprx 50% of GroupedHashAggregateStream work is encoding (variable::encode)
```
physical_plan::aggregates::row_hash::GroupedHashAggregateStream::poll_next
(17,163 samples, **80.95%**)
|-- arrow_row::RowConverter::convert_columns (10,684 samples, **50.39%**)
|--arrow_row::row_lengths (2,080 samples, 9.81%)
|--arrow_row::variable::encode (7,267 samples, 34.28%)
```
<img width="1132" alt="image"
src="https://github.com/apache/datafusion/assets/3950601/82a2986b-808e-4f5b-a198-7a8bc142766a";>
The query is:
```
SELECT count(*), col1, col2, col3, col4
FROM SomeTable
GROUP BY col1, col2, col3, col4
ORDER BY col1, col2, col3, col4
The length of SomeTable is ~150m rows
```
I haven't figured out the root of the problem yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
