Spark on Raspberry Pi?

2014-09-11 Thread Sandeep Singh
Has anyone tried using Raspberry Pi for Spark? How efficient is it to use around 10 Pi's for local testing env ? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-on-Raspberry-Pi-tp13965.html Sent from the Apache Spark User List mailing list archive at

Re: Crawler and Scraper with different priorities

2014-09-08 Thread Sandeep Singh
Hi Daniil, I have to do some processing of the results, as well as pushing the data to the front end. Currently I'm using akka for this application, but I was thinking maybe spark streaming would be a better thing to do. as well as i can use mllib for processing the results. Any specific reason's

Crawler and Scraper with different priorities

2014-09-07 Thread Sandeep Singh
Hi all, I am Implementing a Crawler, Scraper. The It should be able to process the request for crawling scraping, within few seconds of submitting the job(around 1mil/sec), for rest I can take some time(scheduled evenly all over the day). What is the best way to implement this? Thanks. --