Oh interesting solution, a co-worker was suggesting something similar using
resource profiles to increase memory -- but your approach avoids a lot of
complexity I like it (and we could extend it out to support resource
profile growth too).
I think an SPIP sounds like a great next step.
On Tue,
Hello All,
At Uber, we had recently, done some work on improving the reliability of
spark applications in scenarios of fatter executors going out of memory and
leading to application failure. Fatter executors are those that have more
than 1 task running on it at a given time concurrently. This
Hi,
This is my first time using the dev mailing list so I hope this is the correct
way to do it.
I would like to lend my support to this proposal and offer my experiences as a
consumer of spark, and specifically Spark Structured Streaming (SSS). I am more
of an cloud infrastructure devops
