Hi Krishna,
Thanks for your interest contributing to PySpark! I don't personally use
either of those IDEs so I'll leave that part for someone else to answer -
but in general you can find the building spark documentation at
http://spark.apache.org/docs/latest/building-spark.html which includes
Hello,
I am a masters student. Could someone please let me know how set up my dev
working environment to contribute to pyspark.
Questions I had were:
a) Should I use Intellij Idea or PyCharm?.
b) How do I test my changes?.
Regards,
Krishna
1, Yes, because the issues are in JIRA.
2. Nope, (at least as far as MLlib is concerned) because most if it are
just wrappers to the underlying Scala functions or methods and are not
implemented in pure Python.
3. I'm not sure about this. It seems to work fine for me!
HTH
On Fri, Jun 12, 2015 at
Hi,
Thanks for your interest in PySpark.
The first thing is to have a look at the how to contribute guide
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark and
filter the JIRA's using the label PySpark.
If you have your own improvement in mind, you can file your a JIRA,
Hello,
I am currently taking a course in Apache Spark via EdX (
https://www.edx.org/course/introduction-big-data-apache-spark-uc-berkeleyx-cs100-1x)
and at the same time I try to look at the code for pySpark too. I wanted to
ask, if ideally I would like to contribute to pyspark specifically, how
