Re: DataFrame vs RDD

As mentioned earlier, since DataFrame is associated with schema... It makes sense to be created from sqlContext.. So ur statement holds true with that understanding.. On Wed, Mar 23, 2016 at 8:28 AM asethia wrote: > creating RDD is done via spark context where as creating

Re: DataFrame vs RDD

Vinay Kashyap On Wed, Mar 23, 2016 at 7:56 AM Jeff Zhang <zjf...@gmail.com> wrote: > Please check the offical doc > > http://spark.apache.org/docs/latest/ > > > On Wed, Mar 23, 2016 at 10:08 AM, asethia <sethia.a...@gmail.com> wrote: > >> Hi, >> >> I

Re: Enabling spark_shuffle service without restarting YARN Node Manager

applications should not be disturbed. That was the only concern. Thanks and regards Vinay Kashyap On Wed, Mar 16, 2016 at 4:05 PM, Saisai Shao <sai.sai.s...@gmail.com> wrote: > If you want to avoid existing job failure while restarting NM, you could > enable work preserving for NM,

Enabling spark_shuffle service without restarting YARN Node Manager

the shuffle service. Any help would be greatly appreciated. -- *Thanks and regards* *Vinay Kashyap*

Re: Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

Hi Guys, Any help regarding this issue..?? On Wed, Jan 13, 2016 at 6:39 PM, Vinay Kashyap <vinu.k...@gmail.com> wrote: > Hi all, > > I am using *Spark 1.5.1 in YARN cluster mode in CDH 5.5.* > I am trying to create an RDD by reading HBase table with kerberos enabled. >

Error in Spark Executors when trying to read HBase table from Spark with Kerberos enabled

ch is logged which is similar to what I am experiencing *https://issues.apache.org/jira/browse/SPARK-12279 <https://issues.apache.org/jira/browse/SPARK-12279>* Wanted to know if the issue is the same as I am facing..?? And any workaround for the same so that I can proceed with my requirement reading from HBase table.?? -- *Thanks and regards* *Vinay Kashyap*

Re: Low Performance of Shark over Spark.

but not much difference was seen.   Thanks and regards Vinay Kashyap   From:Yana Kadiyska yana.kadiy...@gmail.com Sent:vinay.kashyap vinay.kash...@socialinfra.net Date:Sat, August 9, 2014 6:56 am Subject:Re: Low Performance of Shark over Spark

Support for ORC Table in Shark/Spark

)         at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) FAILED: Execution Error, return code -101 from shark.execution.SparkTask   Any idea how to overcome this..??       Thanks and regards Vinay Kashyap

Low Performance of Shark over Spark.

with Shark over Spark and make it comparable to Impala..??   Thanks and regards Vinay Kashyap

Re: Low Performance of Shark over Spark.

Vinay Kashyap From:Xiangrui Meng men...@gmail.com Sent:vinay.kash...@socialinfra.net Cc:user@spark.apache.org Date:Thu, August 7, 2014 11:06 pm Subject:Re: Low Performance of Shark over Spark. Did you cache the table? There are couple ways

Support for Percentile and Variance Aggregation functions in Spark with HiveContext

with Spark SQL..?? Or .. Are there any limitations still in Spark SQL with respect to UDFs and UDAFs in the version I am using..??     Thanks and regards Vinay Kashyap