Ngone51 commented on a change in pull request #32287: URL: https://github.com/apache/spark/pull/32287#discussion_r618983804
########## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala ########## @@ -613,6 +618,12 @@ final class ShuffleBlockFetcherIterator( } if (isNetworkReqDone) { reqsInFlight -= 1 + if (!buf.isInstanceOf[NettyManagedBuffer]) { + // Non-`NettyManagedBuffer` doesn't occupy Netty's memory so we can unset the flag + // directly once the request succeeds. But for the `NettyManagedBuffer`, we'll only + // unset the flag when the data is fully consumed (see `BufferReleasingInputStream`). + NettyUtils.isNettyOOMOnShuffle = false Review comment: > This flag is in NettyUtils so I am assuming that if there are multiple tasks with their own iterators, then they all will be setting/unsetting the same flag. Is that correct? Correct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org