davehagman edited a comment on issue #3733:
URL: https://github.com/apache/hudi/issues/3733#issuecomment-933845183
> just partitioning on year, month and day did not work out for you and
hence you have to go w/ hour as well?
We tested multiple partitioning schemes and this gave us a
davehagman edited a comment on issue #3733:
URL: https://github.com/apache/hudi/issues/3733#issuecomment-933538622
More details on this issue. The root cause is a when a single batch of
records results in a large number of partitions being scanned for index lookup
(for record de-duplicatio
davehagman edited a comment on issue #3733:
URL: https://github.com/apache/hudi/issues/3733#issuecomment-933538622
More details on this issue. The root cause is a when a single batch of
records results in a large number of partitions being scanned for index lookup
(for record de-duplicatio
