that is preventing further
scaling. Any general advice for doing that kind of detective work would
be
much appreciated.
Thanks,
~ Andrew
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Understanding-Spark-MLlib-failures-tp22641.html
Sent from the Apache Spark
: Understanding Spark/MLlib failures
Hi Andrew,
The .principalComponents feature of RowMatrix is currently constrained to tall
and skinny matrices. Your matrix is barely above the skinny requirement (10k
columns), though the number of rows is fine.
What are you looking to do with the principal
: Thursday, April 23, 2015 4:46 PM
To: Andrew Leverentz
Cc: user@spark.apache.org
Subject: Re: Understanding Spark/MLlib failures
Hi Andrew,
I observed similar behavior under high GC pressure, when running ALS. What
happened to me was that, there would be very long Full GC pauses (over 600
seconds
in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Understanding-Spark-MLlib-failures-tp22641.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr
to identify the bottleneck that is preventing further
scaling. Any general advice for doing that kind of detective work would be
much appreciated.
Thanks,
~ Andrew
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Understanding-Spark-MLlib-failures
-list.1001560.n3.nabble.com/Understanding-Spark-MLlib-failures-tp22641.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional
