This may seem like a silly question, but it really isn’t. In terms of Map/Reduce, its possible to over subscribe the cluster because there is a lack of sensitivity if the servers swap memory to disk.
In terms of HBase, which is very sensitive, swap doesn’t just kill performance, but also can kill HBase. (I’m sure one can tune it to be less sensitive…) But I have to ask how sensitive is Spark? Considering we can cache to disk (local disk) it would imply that it would less sensitive. Yet we see some posters facing over subscription and hitting OOME. Thoughts? --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
