Loading JSON dataset with Spark Mllib

2015-02-15 Thread pankaj channe
Hi, I am new to spark and planning on writing a machine learning application with Spark mllib. My dataset is in json format. Is it possible to load data into spark without using any external json libraries? I have explored the option of SparkSql but I believe that is only for interactive use or

Re: Spark streaming job failing after some time.

2014-11-24 Thread pankaj channe
. Now, I can't figure out as to why it should run successfully during this time even if it could not find SparkContext. I am sure there should be good reason behind this behavior. Anyone has any idea on this? Thanks, Pankaj Channe On Saturday, November 22, 2014, pankaj channe pankajc...@gmail.com

Re: Spark streaming job failing after some time.

2014-11-22 Thread pankaj channe
, Nov 22, 2014 at 8:39 AM, pankaj channe pankajc...@gmail.com wrote: I have seen similar posts on this issue but could not find solution. Apologies if this has been discussed here before. I am running a spark streaming job with yarn on a 5 node cluster. I am using following command to submit my

Spark streaming job failing after some time.

2014-11-21 Thread pankaj channe
I have seen similar posts on this issue but could not find solution. Apologies if this has been discussed here before. I am running a spark streaming job with yarn on a 5 node cluster. I am using following command to submit my streaming job. spark-submit --class class_name --master yarn-cluster