azagrebin commented on issue #7938: [FLINK-10941] Keep slots which contain unconsumed result partitions (on top of #7186) URL: https://github.com/apache/flink/pull/7938#issuecomment-474868838 Thanks for review @zhijiangW and @QiLuo-BD ! As discussed, I agree that consumer confirmation based on transport end would be more performant for certain long running batch consumers. This would require to handle `EndOfPartition` and response with `CancelPartitionPartition` in `RemoteInputChannel` earlier. Now consumer processes all data before sending this response although transport might have already ended. Let’s introduce this option later to avoid bigger changes, it’s a quick fix for older Flink versions anyways and we should ensure at least safety (waiting for consumption is done). I agree that `TASK_MANAGER_RELEASE_WHEN_RESULT_CONSUMED = false` (old behaviour) seems to be useless generally speaking. At the moment I added it more as a safety net for the users to fallback to the old behaviour if something does not work with the newer approach (this quick fix). The idea was to keep the option for some time and drop it later if we see it works or/and after shuffle refactoring where it will most certainly be useless.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
