Holy war is a bit dramatic don't you think? The difference between Scala
and Python will always be very relevant when choosing between Spark and
Pyspark. I wouldn't call it irrelevant to the original question.
br,
molotch
On Sat, 17 Oct 2020 at 16:57, "Yuri Oleynikov (יורי אולייניקוב)" <
I'm sorry you were offended. I'm not an expert in Python and I wasn't
trying to attack you personally. It's just an opinion about what makes a
language better or worse, it's not the single source of truth. You don't
have to take offense. In the end its about context and what you're trying
to
Scala and Python have their advantages and disadvantages with Spark. In my
experience with performance is super important you’ll end up needing to do
some of your work in the JVM, but in many situations what matters work is
what your team and company are familiar with and the ecosystem of tooling
It seems that thread converted to holy war that has nothing to do with original
question. If it is, it’s super disappointing
Отправлено с iPhone
> 17 окт. 2020 г., в 15:53, Molotch написал(а):
>
> I would say the pros and cons of Python vs Scala is both down to Spark, the
> languages in
And you are an expert on python! Idiomatic...
Please do everyone a favor and stop commenting on things you have no idea...
I build ETL systems python that wiped java commercial stacks left and
right. Pyspark was and is and will be a second class citizen in spark
world. That has nothing to do with
I would say the pros and cons of Python vs Scala is both down to Spark, the
languages in themselves and what kind of data engineer you will get when you
try to hire for the different solutions.
With Pyspark you get less functionality and increased complexity with the
py4j java interop compared
