Re: advantages of SparkSQL?

2014-11-25 Thread mrm
Thank you for answering, this is all very helpful! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/advantages-of-SparkSQL-tp19661p19753.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: advantages of SparkSQL?

2014-11-24 Thread Cheng Lian
ache-spark-user-list.1001560.n3.nabble.com/advantages-of-SparkSQL-tp19661.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsub

Re: advantages of SparkSQL?

2014-11-24 Thread Michael Armbrust
; >> Something like: >> data = sqc.parquetFile(path) >> results = data.map(lambda x: applyfunc(x.field)) >> >> Is this faster/more optimised than having the data stored as a text file >> and >> using Spark (non-SQL) to process it? >> >> >> >

Re: advantages of SparkSQL?

2014-11-24 Thread Akshat Aranya
park (non-SQL) to process it? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/advantages-of-SparkSQL-tp19661.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --

advantages of SparkSQL?

2014-11-24 Thread mrm
than having the data stored as a text file and using Spark (non-SQL) to process it? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/advantages-of-SparkSQL-tp19661.html Sent from the Apache Spark User List mailing list archive at Nabble.com