Spark java.lang.SecurityException: class “javax.servlet.FilterRegistration”' with sbt

2015-12-03 Thread Moises Baly
Hi all, I'm having issues with javax.servlet when running simple Spark jobs. I'm using Scala + sbt and found a solution for this error: problem is, this particular solution is not working when running tests. Any idea how can I exclude all conflicted dependencies for all scopes? Here is my partial

Fwd: SparkSQL Postgres balanced partition of DataFrames

2015-07-10 Thread Moises Baly
and was wondering if maybe some of you could clear this points for me, because right now is affecting my cluster performance when processing X million rows and all the heavy lifting goes to one single executor. Thank you for your time, Moises Baly