avamingli commented on PR #1178: URL: https://github.com/apache/cloudberry/pull/1178#issuecomment-2990138113
> The phenomenon showed that the deduplication effect of CBDB’s **Streaming Partial HashAggregate** was significantly worse. As a result, the **Finalize HashAggregate** operator in CBDB processed significantly more data compared to GPDB under the same dataset Thank you for your insights! The Streaming HashAggregate does not spill; it sends out tuples once the hash table is full. Its performance is closely linked to the distinct property of the data from subnodes within the in-memory hash range. However, evaluating this distinct property can be challenging. While HashAggregate may require more space and disk I/O, it can deduplicate more tuples. Conversely, Streaming HashAggregate might incur higher network I/O, deduplicating fewer tuples but potentially speeding up slice execution upon the motion node. Ultimately, the effectiveness of either approach depends on factors like network, disk performance, and, crucially, the distinct property of subnode data, especially given the in-memory limit of the hash table. I encountered a similar observation in #1173, where we discussed adding a GUC to control Streaming HashAggregate. This allows users to tailor the approach to their specific environments, which could be beneficial, especially in ORCA. Anyway, your work looks great! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
