srowen commented on PR #42893: URL: https://github.com/apache/spark/pull/42893#issuecomment-1718045609
I see, you're saying the finalizers are coming from netty, and they're contributing to OOM. I'm curious if something like `-XX:MaxGCPauseMillis` helps your case - turn it down a lot. That, too, has perf implications of course. Do you generally have low memory available to workers / driver? I"m trying to figure out why this doesn't seem to arise regularly, before we try to make a relatively aggressive change here. Running finalizer to _clear memory_ doesn't quite feel right -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
