Recommended cluster parameters

Hi I would like to know the details of implementing a cluster. What kind of machines one would require, how many nodes, number of cores etc. thanks rakesh

Spark SQL shell hangs

Hi I'm trying to convert an XML file to data frame using data bricks spark XML. But the shell hanhs when I do a select operation on the table. I believe it's memory issue. How can I debug this. The cm file sizes 86 MB. Thanks in advance Rakesh Get Outlook for Android

Re: Converting Dataframe to resultSet in Spark Java

Hi Sree I dont think what you are trying to do is correct. DataFrame and ResultSet are two different types. And no strongly typed language will alow you to do that. If your intention is to traverse the DataFrame or get the individual rows and columns then you must try the map function and

Re: My notes on Spark Performance & Tuning Guide

It would be a rare doc. Please share Get Outlook for Android On Tue, May 17, 2016 at 9:14 AM -0700, "Natu Lauchande" > wrote: Hi Mich, I am also interested in the write up. Regards, Natu On Thu, May 12, 2016 at 12:08

Running window functions in spark dataframe

Hi all I am getting hivecontext error when trying to run to run window functions like over on ordering clause. Any help to go about. I am running spark locally Sent from Ouertlook Mobile -- Forwarded message -- From: "King sami"

Error using SQLContext in spark

Error: application failed with exceptionjava.lang.NoSuchMethodError: org.apache.spark.sql.SQLContext.(Lorg/apache/spark/api/java/JavaSparkContext;)V at examples.PersonRecordReader.getPersonRecords(PersonRecordReader.java:35) at

How mature is spark sql

Is it mature enough to use it extensively. I see that it is easier to do than writing map/reduce in java.We are being asked to do it in java itself and cannot move to python and scala. thanksrakesh