Re: Are tachyon and akka removed from 2.1.1 please

I think akka has been removed since 2.0. On 22 May 2017 10:19 pm, "Gene Pang" wrote: > Hi, > > Tachyon has been renamed to Alluxio. Here is the documentation for > running Alluxio with Spark > . > > Hope

Re: Spark app write too many small parquet files

Try limit the partitions. spark.sql.shuffle.partitions This control the number of files generated. On 28 Nov 2016 8:29 p.m., "Kevin Tran" wrote: > Hi Denny, > Thank you for your inputs. I also use 128 MB but still too many files > generated by Spark app which is only ~14 KB

Re: Spark SQL is slower when DataFrame is cache in Memory

t and > 2nd events? > > Best Regards, > Kazuaki Ishizaki > > > > From:Chin Wei Low <lowchin...@gmail.com> > To:Kazuaki Ishizaki/Japan/IBM@IBMJP > Cc:user@spark.apache.org > Date:2016/10/10 11:33 > > Subject:

Re: [Spark] RDDs are not persisting in memory

Hi, Your RDD is 5GB, perhaps it is too large to fit into executor's storage memory. You can refer to the Executors tab in Spark UI to check the available memory for storage for each of the executor. Regards, Chin Wei On Tue, Oct 11, 2016 at 6:14 AM, diplomatic Guru