Hello DB,
Thank you! Do you know how to run Linear Regression without SGD on
streaming data in spark? I tried SGD but due to step size I was not getting
the expected weights.
Best Regards,
Arunkumar
On Wed, Mar 25, 2015 at 4:33 PM, DB Tsai dbt...@dbtsai.com wrote:
Hi Arunkumar,
I think
Hello Jeremy,
Sorry for the delayed reply!
First issue was resolved, I believe it was just production and consumption
rate problem.
Regarding the second question, I am streaming the data from the file and
there are about 38k records. I am sending the streams in the same sequence
as I am reading
Hi Arunkumar,
I think L-BFGS will not work since L-BFGS algorithm assumes that the
objective function will be always the same (i.e., the data is the
same) for entire optimization process to construct the approximated
Hessian matrix. In the streaming case, the data will be changing, so
it will
Regarding the first question, can you say more about how you are loading your
data? And what is the size of the data set? And is that the only error you see,
and do you only see it in the streaming version?
For the second question, there are a couple reasons the weights might slightly
differ,
Hello Jeremy,
Thank you for your reply.
When I am running this code on the local machine on a streaming data, it
keeps giving me this error:
*WARN TaskSetManager: Lost task 2.0 in stage 211.0 (TID 4138, localhost):
java.io.FileNotFoundException:
Hi Arunkumar,
That looks like it should work. Logically, it’s similar to the implementation
used by StreamingLinearRegression and StreamingLogisticRegression, see this
class:
Hello,
I am new to spark streaming API.
I wanted to ask if I can apply LBFGS (with LeastSquaresGradient) on
streaming data? Currently I am using forecahRDD for parsing through DStream
and I am generating a model based on each RDD. Am I doing anything
logically wrong here?
Thank you.
Sample
