Re: Java vs. Scala for Spark

Java 8 lambdas are broken to the point of near-uselessness (because of checked exceptions and inability to close over non-final references). I wouldn't use them as a deciding factor in language choice. Any competent developer should be able to write reasonable java-in-scala after a week and

Re: Java vs. Scala for Spark

It worked for Twitter! Seriously though: scala is much much more pleasant. And scala has a great story for using Java libs. And since spark is kind of framework-y (use its scripts to submit, start up repl, etc) the projects tend to be lead projects, so even in a big company that uses Java the

Java vs. Scala for Spark

All, We're looking at language choice in developing a simple streaming processing application in spark. We've got a small set of example code built in Scala. Articles like the following: http://www.bigdatatidbits.cc/2015/02/navigating-from-scala-to-spark-for.html would seem to indicate that

Re: Java vs. Scala for Spark

Performance wise, Scala is by far the best choice when you use Spark. The cost of learning Scala is not negligible but not insurmountable either. My personal opinion. On Tue, Sep 8, 2015 at 6:50 AM, Bryan Jeffrey wrote: > All, > > We're looking at language choice in

Re: Java vs. Scala for Spark

Thank you for the quick responses. It's useful to have some insight from folks already extensively using Spark. Regards, Bryan Jeffrey On Tue, Sep 8, 2015 at 10:28 AM, Sean Owen wrote: > Why would Scala vs Java performance be different Ted? Relatively > speaking there is

Re: Java vs. Scala for Spark

we are using java7..its much more verbose that java8 or scala examples in addition there sometimes libraries that has no java api, so you need to write them by yourself(e.g. graphx) on the other hand, scala is not trivial language like java, so it depends on your team On 8 September 2015 at

Re: Java vs. Scala for Spark

It's true that Java 8 lambdas help. If you've read Learning Spark, where they use Java 7, Python, and Scala for the examples, it really shows how awful Java without lambdas is for Spark development. Still, there are several "power tools" in Scala I would sorely miss using Java 8: 1. The REPL

Re: Java vs. Scala for Spark

Sean: w.r.t. performance, I meant Scala/Java vs Python. Cheers On Tue, Sep 8, 2015 at 7:28 AM, Sean Owen wrote: > Why would Scala vs Java performance be different Ted? Relatively > speaking there is almost no runtime difference; it's the same APIs or > calls via a thin

Re: Java vs. Scala for Spark

Hi Bryan, I would choose a language based on the requirements. It does not make sense if you have a lot of dependencies that are java-based components and interoperability between java and scala is not always obvious. I agree with the above comments that Java is much more verbose than Scala in

Re: Java vs. Scala for Spark

Why would Scala vs Java performance be different Ted? Relatively speaking there is almost no runtime difference; it's the same APIs or calls via a thin wrapper. Scala/Java vs Python is a different story. Java libraries can be used in Scala. Vice-versa too, though calling Scala-generated classes