dongjoon-hyun commented on PR #60196: URL: https://github.com/apache/spark/pull/60196#issuecomment-6710692085
Regarding the question about G1 heap regions in the review summary: thanks for the analysis. The growing pages are exact powers of two, like the `LongToUnsafeRowMap` pages after SPARK-60998. The same question came up there (https://github.com/apache/spark/pull/60167#discussion_r4499005041), and the page sizes were kept independent of the GC and the memory mode. Region alignment only matters for an on-heap page at least as large as a G1 region, so it adds at most one region for each such growing page of a relation, while the small relations this PR targets use much less heap than the full page they took before. I'd like to keep the page sizes consistent with `LongToUnsafeRowMap` here, and I'll file a new JIRA issue to align the on-heap growing pages of both maps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
