Does feature parity exist between Spark and PySpark

2015-10-06 Thread dant
the duplication, my previous message was held up due to subscription issue. Reposting now. Thanks Dan -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Does-feature-parity-exist-between-Spark-and-PySpark-tp24963.html Sent from the Apache Spark User List mai

Re: Does feature parity exist between Spark and PySpark

2015-10-06 Thread ayan guha
gt; Are there any truth's to the above? I'm a little sceptical. > > Apologies for the duplication, my previous message was held up due to > subscription issue. Reposting now. > > Thanks > Dan > > > > > -- > View this message in context: > http://apache-s

Re: Does feature parity exist between Spark and PySpark

2015-10-06 Thread Richard Eggert
o the above? I'm a little sceptical. >> >> Apologies for the duplication, my previous message was held up due to >> subscription issue. Reposting now. >> >> Thanks >> Dan >> >> >> >>

Re: Does feature parity exist between Spark and PySpark

2015-10-06 Thread Richard Eggert
Spark releases always favour more features being visible and enabled >>> for >>> Scala API than Python API. >>> >>> Are there any truth's to the above? I'm a little sceptical. >>> >>> Apologies for the duplication, my previous message was

Re: Does feature parity exist between Spark and PySpark

2015-10-06 Thread Don Drake
e duplication, my previous message was held up due to > subscription issue. Reposting now. > > Thanks > Dan > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Does-feature-parity-exist-between-Spa

Re: Does feature parity exist between Spark and PySpark

2015-10-06 Thread Siegfried Bilstein
tten >>>> to >>>> Scala to finalise. There are a number of reasons I'm hearing: >>>> >>>> 1. Spark is written in Scala so will always be faster than any other >>>> language implementation on top of it. >>>> 2. Spark releases alwa

Re: Does feature parity exist between Spark and PySpark

2015-10-07 Thread Sean Owen
gt; > Thanks > Dan > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Does-feature-parity-exist-between-Spark-and-PySpark-tp24963.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > ---

Re: Does feature parity exist between Spark and PySpark

2015-10-07 Thread sethah
Platforms-for-Implementing-and-Running-Very-Large-Scale-Machine-Learning-Algorithms-td7823.html#a7824> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Does-feature-parity-exist-between-Spark-and-PySpark-tp24963p24971.html Sent from the Apache Spark User

Re: Does feature parity exist between Spark and PySpark

2015-10-07 Thread Michael Armbrust
> > At my company we use Avro heavily and it's not been fun when i've tried to > work with complex avro schemas and python. This may not be relevant to you > however...otherwise I found Python to be a great fit for Spark :) > Have you tried using https://github.com/databricks/spark-avro ? It shoul