Re: Python level of knowledge for Spark and PySpark

2021-04-14 Thread Mich Talebzadeh
Hi, I believe both Ayan and Jayesh have made valuable points. From my experience the companies that traditionally have a Python team in house would like to add Spark capabilities to their inventory not least because 1. There is more Python code running around that will benefit from modern

Re: Python level of knowledge for Spark and PySpark

2021-04-14 Thread Lalwani, Jayesh
There is no good answer to the question “Have I learnt enough”. You can never learn enough. You have to constrantly learn Practically, if you want to make a career out of using technology XYZ, you only need to learn enough XYZ to get a job doing XYZ. Once you get a job doing XYZ, other people

Re: Python level of knowledge for Spark and PySpark

2021-04-14 Thread ayan guha
The answer always is "it depends". At the outset it seems you are in pretty good shape and have all the key skills you need. All I can suggest is try to take inherent benefits of the language and hone your coding practices On Thu, 15 Apr 2021 at 2:25 am, ashok34...@yahoo.com.INVALID wrote: >

Python level of knowledge for Spark and PySpark

2021-04-14 Thread ashok34...@yahoo.com.INVALID
Hi gurus, I have knowledge of Java, Scala and good enough knowledge of Spark, Spark SQL and Spark Functional programing with Scala. I have started using Python with Spark PySpark. Wondering, in order to be proficient in PySpark, how much good knowledge of Python programing is needed? I know the