Re: [I] Flink API rewriteDataFile How to set up scanning based on file size [iceberg]

2024-01-04 Thread via GitHub
GuoZhaoY closed issue #9386: Flink API rewriteDataFileHow to set up scanning based on file size URL: https://github.com/apache/iceberg/issues/9386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] Flink API rewriteDataFile How to set up scanning based on file size [iceberg]

2024-01-04 Thread via GitHub
GuoZhaoY commented on issue #9386: URL: https://github.com/apache/iceberg/issues/9386#issuecomment-1876786214 Thank you for your answer. I have understood your answer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] Flink API rewriteDataFile How to set up scanning based on file size [iceberg]

2024-01-03 Thread via GitHub
pvary commented on issue #9386: URL: https://github.com/apache/iceberg/issues/9386#issuecomment-187561 If a file is bigger than the TARGET_FILE_SIZE, it will create multiple splits when we read it. The last split of the file is a good candidate to add to merge with a new split, so it co

[I] Flink API rewriteDataFile How to set up scanning based on file size [iceberg]

2023-12-27 Thread via GitHub
GuoZhaoY opened a new issue, #9386: URL: https://github.com/apache/iceberg/issues/9386 ### Query engine flink code : Actions.forTable(executionEnvironment,table) .rewriteDataFiles() .maxParallelism(maxParallelism)