rdblue commented on issue #7843: URL: https://github.com/apache/iceberg/issues/7843#issuecomment-1597517558
This has come up before and the issue is actually that the table has too many files, not that we need to change the design of the queue within `ParallelIterable`. The queue filling up is usually a symptom of needing to compact data in the table. @Heltman can you run a query to summarize the files in your table? How many are there and what is the average size? How many partitions and what's the distribution of file sizes within those partitions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
