General Purpose Spark Cluster Hardware Requirements?

2015-03-08 Thread Nasir Khan
HI, I am going to submit a proposal to my University to setup my Standalone Spark Cluster, what hardware should i include in my proposal? I will be Working on classification (Spark MLlib) of Data streams (Spark Streams) If some body can fill up this answers, that will be great! Thanks *Cores *=

MLlib(Logistic Regression) + Spark Streaming.

2014-12-07 Thread Nasir Khan
I am new to spark. Lets say i want to develop a machine learning model. which trained on normal method in MLlib. I want to use that model with classifier Logistic regression and predict the streaming data coming from a file or socket. Streaming data - Logistic Regression - binary label

Filtering URLs from incoming Internet traffic(Stream data). feasible with spark streaming?

2014-10-27 Thread Nasir Khan
According to my knowledge spark streams uses mini batches for processing, Q: Is it a good idea to use my ML trained Model on a web server for filtering purpose to classify URLs as obscene or benin. If spark streaming handle data as mini batches for processing, will this increase the network

Re: Filtering URLs from incoming Internet traffic(Stream data). feasible with spark streaming?

2014-10-27 Thread Nasir Khan
I am kinda stuck with spark now :/ i already proposed this model in my synopsis and its already accepted :D spark is a new thing for alot of people. what alternate tool should i use now? -- View this message in context:

Machine Learning on streaming data

2014-03-16 Thread Nasir Khan
Level, Kindly Explain in abit detail and if some one can direct me to some good material for me will be greats. Thanks Nasir Khan. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Machine-Learning-on-streaming-data-tp2732.html Sent from the Apache Spark