I had few questions w.r.t to Spark deployment & and way i want to use, It would be helpful if you can answer few.
I plan to use Spark on a embedded switch, which has limited set of resources, like say 1 or 2 dedicated cores and 1.5GB of memory, want to model a network traffic with time series algorithms, the algorithms i want to use currently do no exist in spark, so i'm writing it using R, I plan to use Pipe to get this executed from Spark. The reason i'm using Spark other then the ETL functions is because of portability, so that the same code can be reused on a x86 platform with more CPU & memory resources if required. within the same Application i would like to create multiple threads , one thread doining the testing of ML , and other training at some specific time, and perhaps some other thread to do any other activity if required, Can you please let me know if you see any apparent issues from your experience on spark with this kind of design.