bvaradar commented on issue #2338: URL: https://github.com/apache/hudi/issues/2338#issuecomment-761532496
cc @nsivabalan Not sure if you saw this blog about index usages :https://hudi.apache.org/blog/hudi-indexing-mechanisms/ The stage names could be misleading. It is likely the index lookup is running and not finding the small files. Regarding compaction, I see that there are 413 tasks. Do you have sufficient executors to have them run in parallel. Each task would be compacting the delta files and creating a new parquet file. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
