Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23055#discussion_r237723350
--- Diff: python/pyspark/worker.py ---
@@ -22,7 +22,12 @@
import os
import sys
import time
-import resource
+# 'resource' is a Unix specific module.
+has_resource_module = True
+try:
+ import resource
+except ImportError:
+ has_resource_module = False
--- End diff --
I think it's way better to have the JVM not care and have the python side
handle the setting according to its capabilities.
It's a smaller change, it's more localized, and it reduces the amount of
changes if this needs to be tweaked again in the future.
I don't see the consistency argument you're making. You picked one example
that is very different from this one, since it actually affects the behavior of
the JVM code. Could it have been developed differently? Of course. But we're
not discussing that feature here.
And there are so few of these cases that you really can't draw a pattern
from them.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
