zghong opened a new issue, #67505: URL: https://github.com/apache/doris/issues/67505
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description DUP_KEYS MemTables currently create one `shared_ptr<RowInBlock>` for every inserted row, even though these rows only require their positions for sorting. This introduces substantial per-row metadata overhead, frequent heap allocations, and unnecessary allocation churn during large data loads. The overhead becomes especially significant when a MemTable contains a large number of rows. We can use a more compact representation for DUP_KEYS while preserving the existing sorting semantics: - Sort rows by key in ascending order. - For equal keys, sort by row position in descending order. - Keep the existing behavior for UNIQUE_KEYS and AGG_KEYS unchanged. ### Solution Replace the per-row `shared_ptr<RowInBlock>` objects used by DUP_KEYS MemTables with a contiguous `uint32_t` row-position vector. The row positions are reserved before rows are appended to the mutable block and populated using `std::iota`, avoiding additional allocations after the block has been modified. The existing `RowInBlock` representation remains unchanged for UNIQUE_KEYS and AGG_KEYS because those models require aggregation state. This optimization reduces explicit row-index metadata from at least dozens of bytes per row to 4 bytes per row, eliminates per-row object allocations, and improves MemTable insertion efficiency without changing query or load semantics. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
