[jira] [Comment Edited] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572976#comment-15572976 ] Alessio edited comment on SPARK-15565 at 10/13/16 7:49 PM: --- Yes Sean, indeed in

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572976#comment-15572976 ] Alessio commented on SPARK-15565: - Yes Sean, indeed in my latest issue SPARK-17918 I was referring to

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Environment: Mac OS X 10.11.6 (was: Macintosh) > Default Warehouse location apparently in HDFS >

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehouse location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Summary: Default Warehouse location apparently in HDFS (was: Default Warehause location apparently in

[jira] [Updated] (SPARK-17918) Default Warehause location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Updated] (SPARK-17918) Default Warehause location apparently in HDFS

[ https://issues.apache.org/jira/browse/SPARK-17918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-17918: Description: It seems that the default warehouse location in Spark 2.0.1 not only points at an inexistent

[jira] [Created] (SPARK-17918) Default Warehause location apparently in HDFS

Alessio created SPARK-17918: --- Summary: Default Warehause location apparently in HDFS Key: SPARK-17918 URL: https://issues.apache.org/jira/browse/SPARK-17918 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15565) The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

[ https://issues.apache.org/jira/browse/SPARK-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572759#comment-15572759 ] Alessio commented on SPARK-15565: - Same problem happened again in Spark 2.0.1. > The default value of

[jira] [Commented] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15496312#comment-15496312 ] Alessio commented on SPARK-2352: Dear Sean, I am well aware of the MultiLayer Perceptron classifier. But

[jira] [Commented] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485130#comment-15485130 ] Alessio commented on SPARK-2352: Pretty strange that this post with such hype is still "In progress" after

[jira] [Commented] (SPARK-5575) Artificial neural networks for MLlib deep learning

[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485131#comment-15485131 ] Alessio commented on SPARK-5575: Pretty strange that this post with such hype is still "In progress" after

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333753#comment-15333753 ] Alessio commented on SPARK-15904: - If you're so absolutely sure that I'm missing something, I'm ready to

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Reopened] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio reopened SPARK-15904: - > High Memory Pressure using MLlib K-means > > >

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: *Please Note*: even though the issue has been marked as "not a problem" and "resolved", this

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330054#comment-15330054 ] Alessio commented on SPARK-15904: - For the records, just confirming that the problem is not my code, I've

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327542#comment-15327542 ] Alessio commented on SPARK-15904: - With the --driver-memory 4G switch I've tried both. With no luck. At

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327476#comment-15327476 ] Alessio edited comment on SPARK-15904 at 6/13/16 2:48 PM: -- If anyone's

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327476#comment-15327476 ] Alessio commented on SPARK-15904: - If anyone's interested, the dataset I'm working on is freely available

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327443#comment-15327443 ] Alessio commented on SPARK-15904: - Correct. Memory and Disk gives priority to Memory...but my dataset is

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327438#comment-15327438 ] Alessio commented on SPARK-15904: - My machine has 16GB of RAM. I also tried closing all the other apps,

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327411#comment-15327411 ] Alessio edited comment on SPARK-15904 at 6/13/16 1:55 PM: -- This is absolutely

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327411#comment-15327411 ] Alessio commented on SPARK-15904: - This is absolutely weird to me. I gave Spark 9GB and during the

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327397#comment-15327397 ] Alessio edited comment on SPARK-15904 at 6/13/16 1:48 PM: -- Dear [~srowen], at

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327397#comment-15327397 ] Alessio edited comment on SPARK-15904 at 6/13/16 1:49 PM: -- Dear [~srowen], at

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327397#comment-15327397 ] Alessio commented on SPARK-15904: - Dear [~srowen], at the beginning I noticed that "Cleaning RDD” phase

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327272#comment-15327272 ] Alessio edited comment on SPARK-15904 at 6/13/16 12:45 PM: --- Dear Sean, I must

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327272#comment-15327272 ] Alessio edited comment on SPARK-15904 at 6/13/16 12:44 PM: --- Dear Sean, I must

[jira] [Comment Edited] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327272#comment-15327272 ] Alessio edited comment on SPARK-15904 at 6/13/16 12:41 PM: --- Dear Sean, I must

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327272#comment-15327272 ] Alessio commented on SPARK-15904: - Dear Sean, I must certainly agree with you on k< High Memory Pressure

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: Running MLlib K-Means on a ~400MB dataset (12 partitions), persisted on Memory and Disk.

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327234#comment-15327234 ] Alessio commented on SPARK-15904: - My dataset has 9000+ patterns, each of which has 2000+ attributes.

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Priority: Minor (was: Major) > High Memory Pressure using MLlib K-means >

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Issue Type: Improvement (was: Bug) > High Memory Pressure using MLlib K-means >

[jira] [Commented] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327090#comment-15327090 ] Alessio commented on SPARK-15904: - Hi [~yuhaoyan]], the dataset size is 9120 rows and 2125 columns. This

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: Running MLlib K-Means on a ~400MB dataset (12 partitions), persisted on Memory and Disk.

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: Running MLlib K-Means on a ~400MB dataset (12 partitions), persisted on Memory and Disk.

[jira] [Updated] (SPARK-15904) High Memory Pressure using MLlib K-means

[ https://issues.apache.org/jira/browse/SPARK-15904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessio updated SPARK-15904: Description: Running MLlib K-Means on a ~400MB dataset (12 partitions), persisted on Memory and Disk.

[jira] [Created] (SPARK-15904) High Memory Pressure using MLlib K-means

Alessio created SPARK-15904: --- Summary: High Memory Pressure using MLlib K-means Key: SPARK-15904 URL: https://issues.apache.org/jira/browse/SPARK-15904 Project: Spark Issue Type: Bug