alamb commented on issue #20195: URL: https://github.com/apache/datafusion/issues/20195#issuecomment-4163752112
@gene-bordegaray drew a really nice diagram and explained the problem they were having It seems as if the core issue is that the underlying data is range partitioned, but since there is no way to represent range partitioning in DataFusion, @gene-bordegaray builds a plan that has output partitioning = `Hash(join_key)` even though the underlying data is not actually hash partitioned However, there are some assumptions in the codebase (in this case dynamic filter creation) that rely on the data being actually hash partitioned -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
