Hi, Thanks for that input, I tried doing that but apparently thats not working as well. I thought i am having problems with my spark installation so I ran simple word count and that works, so I am not really sure what the problem is now.
Is my translation of the scala code correct? I don't understand the syntax of scala very well thus wrote my own implementation of streaming kmeans in java, so i am hoping thats correct. Thanks & Regards Biplob Biswas On Sun, Jun 19, 2016 at 3:23 AM, Akhil Das <ak...@hacked.work> wrote: > SparkStreaming does not pick up old files by default, so you need to start > your job with master=local[2] (It needs 2 or more working threads, 1 to > read the files and the other to do your computation) and once the job start > to run, place your input files in the input directories and you can see > them being picked up by sparkstreaming. > > On Sun, Jun 19, 2016 at 12:37 AM, Biplob Biswas <revolutioni...@gmail.com> > wrote: > >> Hi, >> >> I tried local[*] and local[2] and the result is the same. I don't really >> understand the problem here. >> How can I confirm that the files are read properly? >> >> Thanks & Regards >> Biplob Biswas >> >> On Sat, Jun 18, 2016 at 5:59 PM, Akhil Das <ak...@hacked.work> wrote: >> >>> Looks like you need to set your master to local[2] or local[*] >>> >>> On Sat, Jun 18, 2016 at 4:54 PM, Biplob Biswas <revolutioni...@gmail.com >>> > wrote: >>> >>>> Hi, >>>> >>>> I implemented the streamingKmeans example provided in the spark website >>>> but >>>> in Java. >>>> The full implementation is here, >>>> >>>> http://pastebin.com/CJQfWNvk >>>> >>>> But i am not getting anything in the output except occasional timestamps >>>> like one below: >>>> >>>> ------------------------------------------- >>>> Time: 1466176935000 ms >>>> ------------------------------------------- >>>> >>>> Also, i have 2 directories: >>>> "D:\spark\streaming example\Data Sets\training" >>>> "D:\spark\streaming example\Data Sets\test" >>>> >>>> and inside these directories i have 1 file each >>>> "samplegpsdata_train.txt" >>>> and "samplegpsdata_test.txt" with training data having 500 datapoints >>>> and >>>> test data with 60 datapoints. >>>> >>>> I am very new to the spark systems and any help is highly appreciated. >>>> >>>> Thank you so much >>>> Biplob Biswas >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Running-JavaBased-Implementationof-StreamingKmeans-tp27192.html >>>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >>> >>> >>> -- >>> Cheers! >>> >>> >> > > > -- > Cheers! > >
