Managing Dataset API Partitions - Spark 2.0

(newNumberOfPartition)* So I'll be glad to know if there're *any new valuable about custom partitioning dataset, either in the reading phase or at some point?* Thank you so much. Andrea -- *Andrea Spina* N.Tessera: *74598* MAT: *89369* *Ingegneria Informatica* *[LM] *(D.M. 270)

java.lang.OutOfMemoryError Spark MLlib ALS matrix factorization

: spark.executor.memory = 28672m spark.driver.memory = 28672m spark.deamon.memory = 28672m spark.driver.maxResultSize = 0 spark.network.timeout = 3000s Any help will be appreciated. Thank you. -- *Andrea Spina* N.Tessera: *74598* MAT: *89369* *Ingegneria Informatica* *[LM] *(D.M. 270)

Re: Issue with Spark on 25 nodes cluster

Hi, I solved by increasing the akka timeout time. All the bests, 2016-06-28 15:04 GMT+02:00 ANDREA SPINA <74...@studenti.unimore.it>: > Hello everyone, > > I am running some experiments with Spark 1.4.0 on a ~80GiB dataset located > on hdfs-2.7.1. The environment is a 25 nodes

Issue with Spark on 25 nodes cluster

receive signal SIGTERM: 15 I can't figure out a solution as well. Thank you, Regards, Andrea -- *Andrea Spina* N.Tessera: *74598* MAT: *89369* *Ingegneria Informatica* *[LM] *(D.M. 270)