I politely disagree. The jvm is one vm. Python has another. It's less about preference and more about where the skills as an industry is going for data analysis and BI etc. No cares about jvm vs. Pvm. They do care about time. So if the time to prototype is 10x faster (in calendar days) but the VM is slower in cpu cycles, the greater benefit decides what's best. The industry trend is clear now. And seemingly spark is moving in its own direction. In my opinion of course.
Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Sivakumaran S <siva.kuma...@me.com> Date: 9/2/16 4:03 AM (GMT-05:00) To: Mich Talebzadeh <mich.talebza...@gmail.com> Cc: Jakob Odersky <ja...@odersky.com>, ayan guha <guha.a...@gmail.com>, Tal Grynbaum <tal.grynb...@gmail.com>, darren <dar...@ontrenet.com>, kant kodali <kanth...@gmail.com>, AssafMendelson <assaf.mendel...@rsa.com>, user <user@spark.apache.org> Subject: Re: Scala Vs Python Whatever benefits you may accrue from the rapid prototyping and coding in Python, it will be offset against the time taken to convert it to run inside the JVM. This of course depends on the complexity of the DAG. I guess it is a matter of language preference. Regards, Sivakumaran S On 02-Sep-2016, at 8:58 AM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: From an outsider point of view nobody likes change :) However, it appears to me that Scala is a rising star and if one learns it, it is another iron in the fire so to speak. I believe as we progress in time Spark is going to move away from Python. If you look at 2014 Databricks code examples, they were mostly in Python. Now they are mostly in Scala for a reason. HTH Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 2 September 2016 at 08:23, Jakob Odersky <ja...@odersky.com> wrote: Forgot to answer your question about feature parity of Python w.r.t. Spark's different components I mostly work with scala so I can't say for sure but I think that all pre-2.0 features (that's basically everything except Structured Streaming) are on par. Structured Streaming is a pretty new feature and Python support is currently not available. The API is not final however and I reckon that Python support will arrive once it gets finalized, probably in the next version.
