2010YOUY01 commented on PR #22888: URL: https://github.com/apache/datafusion/pull/22888#issuecomment-4736907379
I don't have enough context on the dictionary type support right now, but this is pretty cool. I hope to find some time to look into it later. Here is a bit of context I can share: I assume we would need to specialize a `GroupValues` implementation for dictionary types? Right now, we are going through a larger refactor to change the `GroupValues` contract for better memory efficiency, so I plan to look into dictionary support after the initial blocked-memory patch lands, to save some efforts. I'll @ you when there is progress. TL;DR: today, `GroupValues` allocates memory as a single contiguous vec for all groups. We want to change that to fixed-size blocks. The issue below has more details: - https://github.com/apache/datafusion/issues/7065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
