spark-submit includes a spark-assembly uber jar, which has older versions of many common libraries. These conflict with some of the dependencies we need. I have been racking my brain trying to find a solution (including experimenting with ProGuard), but haven't been able to: when we use spark-submit, we get NoMethodErrors, even though the code compiles fine, because the runtime classes are different than the compile time classes!
Can someone recommend a solution? We are using scala, sbt, and sbt-assembly, but are happy using another tool (please provide instructions how to).
