Spark Processing Large Data Stuck

I run the pagerank example processing a large data set, 5GB in size, using 48 machines. The job got stuck at the time point: 14/05/20 21:32:17, as the attached log shows. It was stuck there for more than 10 hours and then I killed it at last. But I did not find any information explaining why it

How to Run Machine Learning Examples

I want to run the LR, SVM, and NaiveBayes algorithms implemented in the following directory on my data set. But I did not find the sample command line to run them. Anybody help? Thanks. spark-0.9.0-incubating/mllib/src/main/scala/org/apache/spark/mllib/classification -- View this message in

Re: How to Run Machine Learning Examples

: org.apache.spark.mllib.classification.SVM. Program will exit. On Thu, May 22, 2014 at 3:05 PM, Stephen Boesch [via Apache Spark User List] ml-node+s1001560n6278...@n3.nabble.com wrote: There is a bin/run-example.sh example-class [args] 2014-05-22 12:48 GMT-07:00 yxzhao [hidden email]: I want to run the LR, SVM

Re: How to Run Machine Learning Examples

Thanks. I used the following commnad line to run the SVM, but it seems that the path is not correct. What the right path or command line should be? Thanks. ./bin/run-example org.apache.spark.mllib.classification.SVM spark://100.1.255.193:7077 train.csv 20 Exception in thread main

Job Processing Large Data Set Got Stuck

I run the pagerank example processing a large data set, 5GB in size, using 48 machines. The job got stuck at the time point: 14/05/20 21:32:17, as the attached log shows. It was stuck there for more than 10 hours and then I killed it at last. But I did not find any information explaining why it

Re: Job Processing Large Data Set Got Stuck

On Wed, May 21, 2014 at 11:23 AM, yxzhao [hidden email] wrote: I run the pagerank example processing a large data set, 5GB in size, using 48 machines. The job got stuck at the time point: 14/05/20 21:32:17, as the attached log shows. It was stuck there for more than 10 hours and then I killed

Re: How to run the SVM and LogisticRegression

Thanks Xiangrui, But I did not find the directory: examples/src/main/scala/org/apache/spark/examples/mllib. Could you give me more detail or show me one example? Thanks a lot. -- View this message in context:

Re: How to run the SVM and LogisticRegression

: Checkout the master or branch-1.0. Then the examples should be there. -Xiangrui On Mon, May 19, 2014 at 11:36 AM, yxzhao [hidden email] wrote: Thanks Xiangrui, But I did not find the directory: examples/src/main/scala/org/apache/spark/examples/mllib. Could you give me more detail or show me

Re: How to run the SVM and LogisticRegression

, May 19, 2014 at 10:37 PM, Andrew Ash [via Apache Spark User List] ml-node+s1001560n6066...@n3.nabble.com wrote: Hi yxzhao, Those are branches in the source code git repository. You can get to them with git checkout branch-1.0 once you've cloned the git repository. Cheers, Andrew On Mon, May

Measuring Network Traffic for Spark Job

Hi All, I want to measure the total network traffic for a Spark Job. But I did not see related information from the log. Does anybody know how to measure it? Thanks very much in advance. -- View this message in context:

Re: Measuring Network Traffic for Spark Job

is being used and on what machines. https://spark.apache.org/docs/0.9.0/monitoring.html On Tue, Apr 8, 2014 at 12:57 PM, yxzhao [hidden email]http://user/SendEmail.jtp?type=nodenode=3920i=0 wrote: Hi All, I want to measure the total network traffic for a Spark Job. But I did

Measure the Total Network I/O, Cpu and Memory Consumed by Spark Job

Hi All, I am intrested in measure the total network I/O, cpu and memory consumed by Spark job. I tried to find the related information in logs and Web UI. But there seems no sufficient information. Could anyone give me any suggestion? Thanks very much in advance. -- View this

Cannot Access Web UI

http://spark.incubator.apache.org/docs/latest/spark-standalone.html#monitoring-and-logging As the above shows: Monitoring and Logging Spark’s standalone mode offers a web-based user interface to monitor the cluster. The master and each worker has its own web UI that shows cluster and job